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Abstract 



We continue the study of Markov systems started in [5]. In this paper, we 
prove a generalization of Breiman's strong low of large numbers [1] which 
implies a necessary condition for the uniqueness of the stationary state of 
a Markov system. 
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An introduction to Markov systems can be found in one of the previous papers 
by the author, e.g. [5], [6], [7], [8], [9]. 

Let {K,d) be a complete separable metric space. Let M :— (-ft'i(e), We,_Pe)eeB 
be a finite Markov systems on K (see figure 1) with an invariant Borel proba- 
bility measure /i. We assume that iiri(e) is open in K — UeG-B-^«(e)' ^el^i(e) 
continuous and PelKn^y is continuous and bounded away from zero for all e E. 
Obviously, these conditions imply the Feller property of the Markov system. We 
shall denote by Cb{K) the set of all bounded continuous functions on K and by 
P{K) the set of all Borel probability measures on K. Let U : Cb{K) — > Cb{K) 
be the Markov operator associated with the Markov system, given by 



o We foi- all f E Cb{K), 



and U* : P{K) 



P{K) be its adjoint operator, given by 




[/(/) dv for all J eCb{K), vE P{K). 



N = 3 K2 
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Fig. 1. A Markov system. 

We call a Markov system M contractive if and only if there exists < a < 1 
such that 

^^Peix)d{we{x),Weiy)) < ad{x,y) for all x,y e Ki(^e), e€ E. 

eeE 

Let E+ := {(cri,<J2, ...) : ai ^ E, i gN} endowed with the product topology of 
discreet topologies. For x £ K, let Px be the Borel probability measure on S+ 
given by 

-Px(l[ei,...,e„]) :=Pei(a;)Pe2 ° Wei{x)...Pe^ °We„_i O...OWei{x), 

for every cylinder set i[ei,...,e„] := {a G S+ : ai = ei,...,(j„ = e„}, which 
represents the Markov process generated by the Markov system with the Dirac 
initial distribution 6x- 

It was shown in [8] that an irreducible contractive Markov system M with uni- 
formly continuous probabilities Pelx^^) has a unique invariant Borel probability 
measure if Px ^ Py for ah x,y G Kif^^.), e G E, (this was shown in [8] for 
some locally compact spaces, but it holds also on complete separable spaces, as 
contractive A4 also posses invariant measures on such spaces [2]). 

This paper is motivated by the following question. Suppose the Markov system 
Ai has a unique invariant Borel probability measure. Does this imply some 
restrictions on the measures Px, x e K? An answer to this question gives a 
generalization of Breiman's strong low of large numbers [1]. 

Theorem 1 Suppose 'i-/nY^^Zo U''g{x) ^{g) for all x € K , g G Cb{K). Let 
fe : K — »■ [—00, +00] be Borel measurable such that fe\Kn^) is bounded and 



2 



continuous for all e G E. Then, for every x € K , 



1 ""^ /■ 
lim - V f„ o o ... o (x) = V / pefe djj 

fc=0 eeEj/ 

for Px-a.e. a e S+. 

Proof. We climb on the shoulders of Brciman [1]. By Kakutani-Yosida norms 
ergodic lemma (e.g. see [3] p. 441), ^/nY^^^^U^'g converges uniformly to 
Jgdu for all g G Cb{K). Therefore, UHPefe) converges uniformly 

to J Pefe dfj, for all e G E (we use the convention x oo = 0). 

Now, fix X £ K and define 



X, n = 0, 



forallc7eS+,neN, 



1.^ r x;-i;(x;|x„_i), n>i 

" ■ 1 0, n < 1, 



and 

£;(x;|x„_fc+i)-i?(x;|x„_fc), n>fc 

0, n < fc 

for k > 1, where E(.|.) denotes the conditional expectation with respect to 
measure Px- 

We are going to use the following result ([3], p. 387). Let Yi, Y2, ... be a sequence 
of random variables such that E{Yn\Yn-i, ■■■,Yi) = and EY^ < c < 00 for all 
n. Then 'i-fmY^^i Vj — > a.s.. 

To apply this, note that 

E{^n\^n-l^ ^1) = E{E{Zl^\Xn-k, Xn-k-1, Xi)\Z^_-^, Z^), 

and that, since the Xi,X2,... form a Markov chain, 

E{Z^\Xn-k,Xn-k-l, •••,-'^1) = E{Z^\Xn-k) = 0. 

Furthermore, E{Z!^Y < 2maxee£; ||/e|i<-i(,, Thus, 

-^Zf^O P,-a.e.. (1) 

1=1 

Now, write 

X'^-E{X'^\Xn-k) = Zl + Zl + ... + Zl n>k. 
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Thus, by (1), 



lim 



^ m ^ rn 



Or, neglecting at most k terms, 



Px-a.e.. 



lim 

m— *C30 



^ m ^ m 



n=l 



n=l 



= P^-n.e.. 



So that, for fixed s, 



lim 

m— *oo 



n=l n=l \ k=l / 



Px-a-e.. 



(2) 



Now, observe that 

Yl Pen+i{Xn)Pen+2{X„)... 

e„ + i,...,e„+fc 



Since ^{Pefe) converges uniformly to / Pe/e c^M for all e G E, for 

every e > 0, we can choose s > such that 



e. 



By (2), for such s, 



lim 

m—*oo 



1 ™ /■ 
-YK-Yjpefe 



djjL 



k=l eeE 



< e Px-a-e.. 



□ 



Corollary 1 Suppose K is a compact metric space and fi is a unique invariant 
Borel probability measure of the Markov system A4. Let fe-K^ [— oo, +00] be 
Borel measurable such that felxn^^ is continuous for all e £ E. Then, for every 
X G K, 

1 f 

lim - V O O ... O Wa^ {x) = y] / Pe/e 

n— »oo n ^ I 
k=0 e^Er/ 

for Px-a.e. a G S+. 
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Proof. By the hypothesis P{K) is weakly* compact. Let x G K . Since every 
weakly* convergent subsequence l/^mX]fc=o~ U*^5x converges to an invariant 
Borel probability measure of U* , by the uniqueness of jj., we conclude that 

n ^-^ 

k=0 

Hence, the claim follows by Theorem 1. □ 



Remark 1 Obviously, Corollary 1 is a generalization of Breiman's strong low 
of large numbers [1] for finite Markov systems. 

The following is a version for contractive Markov systems. 

Corollary 2 Suppose K is a complete separable metric space. Suppose is a 
contractive Markov system with a unique invariant Borel probability measure jjb. 
Let fe'.K-^ [—oo, +oo] be Borel measurable such that felK^^^ is bounded and 
continuous for all e G E. Then, for every x € K , 

1 f 

n—>oo n ' ^ ' / 

fe=0 eeEj/ 

for Px-a.e. a S S"*". 



Proof. It was shown in [2] that the hypothesis impHes 

n-1 

n 



k=0 

for all X G K, g G Cb{K). Thus, the claim follows by Theorem 1. □ 



Definition 1 Let S := {(..., (t_i, o-Qi ""ij •••) ■ Ci G E \/i G Z} cndowod with the 
product topology of discreet topologies. Denote by S the left shift map on S. 
We call the shift invariant Borel probability measure M on S given by 

M {rn[ei,...,ek]) := j Pei{x)Pe2{WeiX)...Pe^{Wek_i O ... O WeiX)dn{x), 

for every cylinder set m[^i, •••) ^k] := {a G T, : ai = Ci for all m < i < n}, the 
generalized Markov measure (see [6]). We call the measure preserving transfor- 
mation S : (I],M) — > (E,M) a generalized Markov shift. 



The following corollary gives a necessary condition on measures Px, x G K, for 
the uniqueness of jj.. 
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Corollary 3 Suppose /i is a unique invariant Borel probability measure of A4. 
Suppose K is a compact metric space, or K is a complete separable metric space 
and M. is contractive. Then, for every x €: K, 



lim -logP^(l[cri, ...,(7„]) = V / Pelogp, 
n—yoo n ^ — ' / 



for Px-a.e. a G 



Proof. Set /e := logPe for all e € E. Then, by Corollary 1 or Corollary 2, for 
every x G K, 

lim - {x)pa2 O {x)...Pa^ O Wa„_i O ... O 1/;^^ {x)] = / logPe dfj. 

n— »(x> n '—^ I 

for P^-a.e. cr e E+. □ 

Remark 2 Note that —J2eeE Ik Pe^ogp^ djjL is the Kolmogorov-Sinai en- 
tropy of the generalized Markov shift associated with M. and /i (this was proved 
in [9] under additional assumption of uniform continuity of each Pe\Kn^)). This 
entropy formula also plays a central role in [4]. 
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